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ABSTRACT 


s'  It  is  a  result  of  Mangasarian  that  the  nonlinear  complementarity  prob¬ 
lem  is  equivalent  to  the  solution  of  a  system  of  nonlinear  equations.  This 
paper  considers  the  solution  of  this  system  by  the  damped  Gauss-Newton 
method  using  the  priniciple  of  least  squares  minimization.  Algorithms  are 
developed  which  under  fairly  simple  conditions  converge  locally  to  a  solution 
and  under  stronger  hypotheses  converge  locally  quadratically.  The  algo¬ 
rithms  have  been  used  successfully  to  solve  both  linear  and  nonlinear  com¬ 


plementarity  problems  including  a  standard  difficult  test  problem  due  to 


Colville. 
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AMS(MOS)  Classification:  90C30,  90C25 
Keywords:  Gauss-Newton  method,  Complementarity  problem 
Work  Unit  Number  5:  Optimization  and  Large  Scale  Systems 
Sponsored  by  the  United  States  Army  under  Contract  No.  DAAG29-80-C- 
0041.  This  material  is  based  on  work  sponsored  by  National  Science  Foun¬ 
dation  Grants  DCR-8420963  and  MCS-8102684. 


SIGNIFICANCE  AND  EXPLANATION 

Present  methods  of  solution  for  the  nonlinear  complementarity  problem 
consist  of  fixed  point  methods  and  iterative  linearization.  This  paper  pro¬ 
poses  solution  of  the  nonlinear  complementarity  problem  by  solving  a  system 
of  nonlinear  equations  due  to  Mangasarian.  Computational  experience  shows 
that  the  method  is  viable  and  presents  an  alternative  approach. 


The  responsibility  for  the  wording  and  views  expressed  in  this  descriptive 


summary  lies  with  MRC,  and  not  with  the  author  of  this  report. 
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GAUSS-NEWTON  METHODS 
FOR  THE  NONLINEAR  COMPLEMENTARITY  PROBLEM 


P.  K.  Subramanian 


1.  Introduction 

Given  an  operator  F:  5Rn  — *  9£n  ,  the  complementarity  problem  consists 
in  finding  a  z*  (if  it  exists)  such  that 

2*  >  0,  F{  z*  )  >  0  and  2*  T F(  2*  )  =  0. 

When  F  is  an  affine  operator,  F  =  Mx  +  q  where  M  is  an  n  x  n  matrix  and 
q  €  $Rn  ,  the  complementarity  problem  is  referred  to  as  the  linear  comple¬ 
mentarity  problem  and  denoted  by  LCP(M ,  q) .  When  F  is  not  affine,  the 
complementarity  problem  is  referred  to  simply  as  the  nonlinear  complemen¬ 
tarity  problem  and  we  shall  write  NLCP(F)  in  this  case.  The  complemen¬ 
tarity  problem  arises  in  many  situations  such  as  equilibrium  problems  as  well 
as  optimality  conditions  for  mathematical  programming  problems  (Cottle  & 
Dantzig,  [1968]). 

Sponsored  by  the  United  States  Army  under  Contract  No.  DAAG29- 
80-C-0041.  This  material  is  based  on  work  sponsored  by  National  Science 
Foundation  Grants  DCR-8420963  and  MCS-8102684 
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Attempts  to  solve  the  linear  complementarity  problem  have  met  with 
remarkable  success  when  M  is  positive  semidefinite  or  has  other  properties. 
Although  the  general  case  of  LCP(M ,  q)  is  NP-complete  (Chung  [1979]), 
robust  methods  have  been  found  for  special  cases.  Both  direct  and  iterative 
methods  are  available.  The  best  known  direct  methods,  which  are  based 
on  the  process  of  pivoting  are  due  to  Lemke  [1965]  and  Cottle  &  Dantzig 
[1968].  Typically,  these  methods  end  in  a  finite  number  of  steps.  The  iter¬ 
ative  methods  are  infinite  methods  producing  a  sequence  of  iterates  whose 
accumulation  point (s),  if  they  exist,  solve  the  problem.  Well  known  amongst 
such  mehtods  are  those  of  Cryer  [1971],  Cottle  et  al  [1978]  and  Mangasari&n 
[1977]. 

On  the  other  hand,  methods  of  solution  for  nonlinear  complementarity 
problem  are  essentially  iterative  in  nature.  The  best  known  are  the  simplicial 
division  methods  or  the  the  so  called  path  following  methods  due  to  Scarf 
[1967,  1973],  Todd  [1976]  and  Eaves  [1972].  In  such  methods  one  essentially 
computes  the  fixed  points  of  appropriate  continuous  maps  such  as 

x  *-+  (i-ir(i))+  :=  min{0,x  -  F(x)} 

where  x+  is  the  projection  of  x  on  the  nonnegative  orthant. 

If  F  is  differentiable,  given  the  iterate  xk,  we  can  consider  the  lineariza¬ 
tion  Ck{F)  of  F  at  the  point  xk  : 

£k{F)x  =  F(xk)  +  VF(xk)(x  -  xk) 

=  F{xk)  -  VF(xk)xk  +  VF(xk)x 
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where  VF(ifc)  is  the  Jacobian  of  F  at  xk.  Hence  if  we  let 

Mk  :=  VF(xk),  qk  :=  F(xk)  -  VF(xk)xk, 

we  get  a  sequence  {xk}  of  iterates  such  that  xk+1  solves  LCP(Mk,qk).  Using 
results  from  the  theory  of  generalized  equations  developed  by  Robinson  [1976, 
1978,  1979,  1980],  Josephy  [1979]  has  shown  that  under  suitable  hypotheses, 
the  iterates  converge  locally  quadratically  to  a  solution  x*  of  NLCP(F)  , 
thus  reducing  NLCP(F)  to  a  sequence  of  linear  complementarity  problems. 
He  has  used  this  method  in  particular  to  solve  Hogan’s  PIES  model  (Hogan 
[1975]  using  Lemke’s  algorithm  to  solve  the  resulting  LCPs. 

Our  approach  in  this  paper  is  different.  In  an  interesting  paper,  Man- 
gasarian  [1976]  established  that  NLCP(F)  is  equivalent  to  the  solution  of 
a  system  of  nonlinear  equations  G{z)  =  0  for  a  suitably  defined  operator 
G:  5Rn  — ♦  5Rn  .  We  shall  consider  several  algortihms  in  this  paper  to  min¬ 
imize  g{z)  =  ||G(z)||2/2.  Thus  our  algorithms  are  somewhat  in  the  spirit 
of  least  squares  minimization  and  our  methods  are  variants  of  the  so  called 
damped  Gauss- Newton  procedures.  We  shall  show  that  under  fairly  simple 
conditions  these  algorithms  exhibit  local  superlinear  convergence  and  un¬ 
der  stronger  hyptheses,  local  quadratic  convergence.  Like  all  Newton-type 
methods,  these  methods  are  expensive  since  one  needs  to  solve  a  system  of 
linear  equations  at  each  step  to  find  the  Gauss-Newton  direction.  Neverthe¬ 
less,  used  in  conjunction  with  cheaper  iterative  methods  to  produce  suitable 
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starting  points,  these  methods  may  prove  viable.  We  prefer  to  use  the  Gauss- 
Newton  procedure  vis-a-vis  the  Newton  procedure  since  we  do  not  require 
the  Jacobian  of  G  to  be  invertible  at  each  iteration.  In  one  version  of  our 
algorithm,  it  turns  out  that  when  VG  is  invertible  our  method  reduces  to 
the  Newton  method. 

We  briefly  describe  the  notation  used  in  this  paper.  We  use  5Rre  for  the 
space  of  real  ordered  n-tuples  .  All  vectors  are  column  vectors  and  we  use  the 
Euclidean  norm  throughout.  Given  a  vector  x,  we  denote  its  ith  component 
by  X,.  We  say  x  >  0  if  x,  >  OVi.  The  nonnegative  orthant  is  denoted  by 

. 

We  use  superscripts  to  distinguish  between  vectors,  e.g.,  x*,x2  etc.  For 
x,y  €  3?n  xT  indicates  the  transpose  of  x,  xTy  their  inner  product.  Occa¬ 
sionally,  the  superscript  T  will  be  suppressed.  All  matrices  are  indicated  by 
upper  case  letters  A,  B,  C  etc.  The  ith  row  of  A  is  denoted  by  At  while  its 
jt,lcolumn  is  denoted  by  A.j.  The  transpose  of  A  is  denoted  by  AT. 

Real  valued  functions  defined  on  subsets  of  are  denoted  by  lower 
case  letters  /,  g,  6  etc.,  and  we  write  V/(x)  and  V2/(x)  to  indicate  the 
gradient  vector  and  the  Hessian  matrix  of  /  at  the  point  x.  If  F  is  an 
operator,  F  :  — *•  9?m,  we  shall  write  VF  to  indicate  the  m  x  re  Jacobiar*. 

matrix  of  F  at  a  given  point. 

Given  N LC P[F)  ,  we  define  the  feasible  set  and  solution  set  by  S{F) 
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and  S(F)  respectively,  that  is, 

5(F)  =  {ier+  :F(x)  6  3^} 

S(F)  =  {i£  S(F)  :  xtF(x)  =  0}. 

In  the  case  of  LCP{M,  q) ,  we  shall  denote  these  sets  by  S(M,q)  and  S(M,q) 
respectively.  Finally  the  end  of  a  proof  is  signified  by  |. 

2.  Nonlinear  equations  and  NLCP(F) 

Our  starting  point  for  this  paper  is  the  following  theorem  due  to  Mangasarian 
[1976]  which  shows  the  equivalence  of  NLCP(F)  to  the  solution  of  a  system 
of  nonlinear  equations. 

2.1  Theorem.  Let  F :  3?n  — ►  3 ln  ,  0  a  strictly  increasing  function  such 
that  6  :  5R  — ►  3?  with  0(0)  =  0 .  Then  z*  solves  NLCP(F)  if  and  only  if  z * 
solves  the  system  of  nonlinear  equations  G,(z)  =  0  t  =  1,  2  n  where 

g.W  =  -  2.1)  -  «(*(*))  -  »(*) 

and  F,(z)  is  the  ith  component  of  F(z) . 

We  shall  call  G(z)  the  M-function  associated  with  F .  Mangasarian 
also  proved  the  following  corollary  which  we  shall  find  particularly  useful 
later. 

2.2  Corollary.  Suppose  that  z*  solves  N LC P(F)  .  Assume  further  that 
F  is  differentiable  at  z*  , 

(1)  z*  is  nondegenerate,  that  is  z*  +  F(z')  >  0, 


(2)  VF(2*) ,  the  Jacobian  of  F  at  z*  has  nonsingular  principal  minors 
and 

(3)  0  is  differentiable,  strictly  increasing  such  that 

f  >  0  =>  0'(f)  +  0'(O)  >  0. 

Then  2*  solves  G(z)  =0  and  the  Jacobian  VG(z*)  of  G  is  nonsingular. 

3.  A  Gauss-Newton  algorithm 

Let  F:  9?"  — >  9?"  and  consider  NLCP(F)  .  For  the  rest  of  this  section 
we  shall  assume  that  VF(z)  exists  and  the  function  0(c)  in  Theorem  2.1  is 
taken  to  be  0(f)  =  f|f| .  Let  G(2)  be  the  associated  M-function.  Define 

*(*)  =  |l|G(*)ll*- 

We  are  going  to  develop  an  algorithm  to  minimize  g ,  somewhat  in  the  spirit 
of  least  squares  minimization.  We  note  that  z*  solves  G(z)  =  0  if  and  only 
if  it  is  a  global  minimizer  of  g .  Also,  if  z*  is  a  critical  point  of  g  and 
VG(z*)  is  nonsingular,  then  G(z*)  =  0  since 

Vff(2*)  =  VG(2fG(2*). 

In  this  case,  from  Theorem  2.1,  z*  solves  NLCP(F)  .  Hence  our  aim  is  to 
find  algorithms  to  find  critical  points  of  g. 
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Given  s  €  9in  ,  let  us  linearize  G  about  s  and  consider  (cf.  Ortega  & 
Rheinboldt,  [1970],  page  267) 

».(*)=  1||G(*)  +  VGW(*-.)||J. 

Then  the  gradient  Vgs(x)  with  respect  to  x  of  gs  is  given  by 
Vg3(x)  =  VG(s)t{G(s)  +  VG(i)(i  -  a)). 

Hence  the  Hessian  Hs(x)  of  gs  is  given  by 

Hs{s)  =  VG(s)rVG(s)  =:  A,  (3.1). 

We  note  that  As  is  positive  semideSnite  and  symmetric. 

3.2  Lemma,  Let  A  >  0.  For  any  x  e  D?n  ,  let  Az  —  VG(z)TVG(~) 
Suppose  that  Vg(x)  ^  0.  Then  the  direction  p  given  by 

(Ax  +  A  l)p  =  Vg(  x) 

is  an  ascent  direction  for  g.  In  particular,  3w  >  0  such  that  g(x  —  wp)  < 
g{x). 

Proof 

There  exist  constants  -7  >  0  and  u  >  0  such  that 


'y||/i|]2  <  hT Azh  <  v\\h\\2 


V  h  e  5»n. 


It  follows  that 


(T  +  X)\\h\\2  <  hT{Ax  +  XI)h  <  (u  +  A)||h||2  Vfc  <E  »n. 

Since  Vg(x)  ^  0,  p  ^  0.  If  we  take  h  =  p,  we  get 

pTVg{x)  >  (t  +  A)||p||2  >  0. 

It  follows  that  Vg(x)  •  p  >  0  and  hence  [Ortega  and  Rheinboldt,  1970, 
8.2.1]  that  p  is  an  ascent  direction  for  g .  | 

3.3  Damped  Gauss-Newton  Algorithm 

The  Lemma  just  proved  leads  us  to  consider  the  following  algorithm. 

(1)  Let  x°  be  given.  Having  xk ,  determine  x*+1  as  follows  : 

(2)  If  Vg(xk)  =  0,  stop. 

(3)  If  V</(xfc)  ^  0,  define 

Ak  =  VG(xfc)TVG(xfc), 

Xk  =  g(xk)  and 

pk  =  (Ak  +  XkI)-lVg(xk). 

Let  Wfc  be  the  largest  element  in  the  set  fl  =  {l,  1/2,  •••  1  /2n,  *  -  -} 
such  that  g(xk  —  ukpk)  <  g(xk) .  Set 
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3.4  Theorem.  Let  x°  be  given  and  let  {xfe}  the  sequence  determined  by 
Algorithm  S.S.  Assume  that 

(3.5)  sup*  ||VG(xfc)||  <  oo  and 

(3.6)  Vgr(x)  is  Lipschitzian  with  modulus  K  on  D?”  . 

Then  either  {xk}  terminates  at  a  stationary  point  of  g  or  else  every  limit 
point  of  {xfe}  ,  if  it  exists,  is  a  stationary  point  of  g. 


Proof 

The  first  assertion  is  obvious.  Let  0  <  6  <  1 .  Let  7*  be  the  smallest 
eigenvalue  of  Ak  .  We  claim  that 


(3.7) 


If 


Wfc  > 


2 (A*  +  7fc)(l  ~  b) 
K 


then  (3.7)  holds  trivially.  Assume,  therefore,  that 


Wfc  < 


2(Afc  +  7fc)(l  ~  6) 

K 


(3.8) 


By  our  assumptions  on  Vg(x), 


II Vg{y)  -  Vg(z)|j  <  K\\y  -  z\\ . 


Hence  [Ortega,  1972,  p  144], 


g(xk  -  u>kpk)  -  g(xk)  +  Vg(xk)u>kpk 


<  T-uilipV 
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Hence  (3.8)  implies  that  g(xk)  >  g(xk  -  UkPk)  •  Since  w*  is  the  largest 
weH  chosen  to  satisfy  p(xfc)  >  g(xk  -  ujpk) ,  it  follows  that  2w*  violates 
(3.8)  so  that  (3.7)  holds.  This  proves  our  claim. 

Assume  now  that  x*  is  a  limit  point  of  {xfc}  and  that  xkj  — +  x*  .  From 
(3.11)  we  have 

g{xk>)  -  g(xk^‘)  >  g(xk>) -g(xk’  +  1) 

.  &*JV*(x*')ll8 

(A*,  +  7) 

— ^  0.  (3.12) 

If  liminfw^  =  0  then  by  (3.7), 

0  =  liminf  A*,  =  Jim  inf  g[xki)  =  }\mg{xk>)  =  g(x*), 

that  is  G(x*)  =  0  and  hence  Vg(x*)  =0.  If,  however,  lim  inf  = 
w*  >  0,  then  from  (3.12)  and  the  continuity  of  Vg(x),  ||ff(x*)||  =  0,  that 
is  Vg(x*)  =  0.  | 

4.  Local  superlinear  convergence 

In  this  section  we  shall  investigate  situations  under  which  conditions  (3.5) 
and  (3.6)  of  Theorem  (3.4)  can  be  realized.  We  shall  then  prove  that  under 
appropriate  conditions,  Algorithm  (3.3)  exhibits  local  superlinear  conver¬ 
gence.  We  begin  with  the  notion  of  a  locally  Lipschitzian  operator. 

4.1  Definition.  An  operator  F:  3?n  — ♦  9?n  is  said  to  be  locally  Lipschitztan 
at  a  point  z  if  3  K  >0  and  a  neighborhood  N  =  N(z)  of  z  such  that  F 
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is  Lipschitzian  on  N  that  is 

l|F(x)  -  F(j/)||  <  K\\x-y\\  Vx,  y  £  N 
and  we  write  F  £  L(N). 

4.2  Lemma.  Let  F :  9?n  — ♦  9f"  and  consider  NLCP(F)  .  Let  G:  3?"  — ► 
5Rn  be  the  associated  M-function,  g(x)  =  ||G(x)||2/2 .  Suppose  further  that 
VFi(z),  i  =  1,  2,  •••  n  is  locally  Lipschitzian  at  some  point  z.  Then  35  > 
0  such  that  Vg  is  Lipschitzian  on  S ,  where 

S  =  S(z,  6)  =  {x  :  ||x  -  z\\  <  5}. 


Proof 

Recall  that  for  *  =  1,  2,  •  •  •  n 


Gi(z)  =  {F,(z)  -  *}2  -  F,(2)|F(2)|  -  *|*|  (4.3) 


so  that 

VGi(z)=2(Fi(z)-zi)(VFi(z)-ei) 

-2|F,(2)|VF,(*)-2|*|*.  (4.4) 

Since  VFi(z)  £  L(S) ,  it  is  continuous  and  so  are  VF(2),  VG*(z)  and 
VG(2).  Hence  ail  these  operators  are  bounded  on  S.  This  implies  that 
Fi{z ),  Gi{z )  and  G{z)  £  L(S) . 


•f. 


~  .■* . "•  -f-V'  j.*"  -*•  •  ■  **  »"•  ■■ 
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Now  if  A,  B  :  D  C  3?n  -*  »m  and  A,  B  €  L(D) ,  it  is  trivial  that 
A  ±  B  e  L(D) .  However,  if  A  and  B  are  also  bounded  on  D ,  it  follows 
from  the  identity 

II A(x)B(x)  -  A(y)H(y)|| 

=  i||  (A(x)  +  A(y))(B[x)  -  B{ »))  +  (B(i)  +  B(»))(A( i)  -  A(y))|| 

that  AH  €  L(5) .  From  (3.4)  we  now  deduce  that  VG,(z)  €  L(S)  and 
hence  that  VG(z)  6  L(S) .  Similar  considerations  show  that  V<j(z)  = 
VG(z)rG(z)  e  L(S).  | 

4.5  Theorem.  Let  F:  3?n  — ♦  SRn  and  consider  NLCP(F)  .  Let  G:  5Rn  — + 
SR”  be  the  associated  M-f unction.  Let  z*  solve  NLCP{F)  ,  V Ft(z)  be  lo¬ 
cally  Lipschitzian  at  z *,  1  <  i  <  n.  Assume  that  the  hypotheses  of  Corol¬ 
lary  (2.2)  are  satisfied.  Then  there  exists  a  neighborhood  N(z*)  such  that  if 
x°  6  N(z*)  then  the  sequence  {x*}  defined  by  Algorithm  (8.8)  satisfies  : 

(1)  xk  €  N{z*),  xk  — >  z* , 

(2)  Conditions  (8.5)  and  (8.6)  of  Theorem  8.4  hold  and 

(3)  If  K  is  the  constant  given  by  (8.6),  7*  the  smallest  eigenvalue  of 
VG(z*)tVG(z*)  and  27*  >  K  then  xk  — *  z*  superlinearly. 

Proof 

By  Corollary  2.2,  VG(  z * )  is  nonsingular  and  by  Theorem  2.1,  G(  z*  )  =  0 . 
Let  Hi,  H2,  •  •  •  Hn  be  the  eigenvalues  of  VG(  z*  ) ,  £  =  {m  '•  Re(p  i )  <  0} 


-  *  -  *'•  L'“  »  - 

-  .  * 

-a  -  .li.U.’.  mi 


where  Re(n)  denotes  the  real  part  of  n .  Let 

(  min  {  — |/x,|2/i2e(^,)  :  m  e  £} 


{  +oo,  if  E  =  0  . 

We  can  find  6  >  0  such  that  for  all  z  €  S  =  S(  z*  ,  6)  =  {x  :  \\x  -  z*  ||  < 

6},  we  have  g(z)  =  ||G(z)||2/2  stnd  tiliztti  V is  Lipschitzian  with 

modulus  K  on  S  (Lemma  4.2). 

Let  xk  be  in  5 .  In  Algorithm  3.3,  0  <  u>k  <  1  by  choice  and  since 
A k  =  d(xk)  <  V  ,  it  follows  that 

0  <  u  <  1,  0  <  -k-  <  v. 

2  - 

It  is  a  consequence  of  [Ortega  and  Rheinboldt,  10.2.3]  that  z*  is  a  point  of 
attraction  of  {i*}  and  that  xk  -*  z* .  By  Lemma  4.2  and  our  choice  of  S , 
it  is  clear  that  (3.5)  and  (3.6)  hold. 

Assume  now  that  7*  >  K/2.  Recall  that  w*  is  the  largest  element  in 
fl  such  that  g(xk  -  w*pfc)  <  g(xk) .  We  saw  in  the  proof  of  Theorem  3.4 
that 

2(Afc  +  7fc)  ^  /  k\ 

w  ^ -  =>  -  “P  )  <  ff(*  )• 

Since  At  -+  0,  it  follows  that  2(A*  +  lk)/K  -*■  27*  >  1  and  hence  that 
w*  — »  1 .  Thus,  A*  — ♦  0,  u;*  — »  1  and  by  [Ortega  and  Rheinboldt,  pl24] 
that  xk  — ►  z*  superlinearly.  | 

5.  Local  quadratic  convergence 

Let  x°  be  given.  If  we  assume  that  VG(z)  satisfies  hypothesis  stronger  than 


’■**•'.**  *  .  •  V  V  •.•*.*  »  - 

•>  *.■  *•*.**  v  *.»  -  •V" 


v  *.  •*  v.v. 


(3.6),  we  can  modify  Algorithm  3.3  so  that  the  resulting  iterates  converge 
locally  quadratically. 

5.1  Modified  Gauss-Newton  Algorithm 

Consider  the  following  algorithm  in  which  the  perturbation  parameters  A* 
are  chosen  slightly  differently: 

(1)  Let  x°  be  given.  Having  xk ,  define  x*+1  as  follows: 

(2)  If  Vg(xfc)  =  0,  stop. 

(3)  If  Vg(xk)  #  0,  let 

Afc  =  VG(xfc)rVG(xfc), 

7*  =  smallest  eigenvalue  of  A*. 

(4)  Define  the  A*  by 

{0,  if  ~fk  >  0; 
ff(x*),  otherwise  . 

(5)  Set  pk  =  ( Ak  +  Afc/)-1  Vg(xfc). 

(6)  Let  ojk  be  the  largest  element  in  Q  =  {1,  1/2,  ••  •}  such  that 
g{xk  -  <jJkPk)  <  g(xk). 

(7)  Set  xk+1  =  xk-ukpk. 

5.2  Theorem.  Let  F :  5Rn  — ♦  5Rn  and  assume  that  x°  is  given .  Let  {xfc} 
be  the  sequence  of  iterates  of  Algorithm  5.1.  Assume  that  : 

(5.3)  0  <  liminffc  ||VG(xfc)||  <  sup*  ||VG(xfc)||  <  oo  and 
(5.4)  Vg(x)  is  Lipschitzian  with  modulus  K  on  5Rn  . 


m"«  y  ."a-.'  rv'trj  V.-  ITVV-J  V.  w  if  j  v 


4.^  i  4."  •"*  a:"  s "  ,;1*'  ;■*■’.■■  v 
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Then  either  {xk}  terminates  at  a  stationary  point  of  g,  or  else  every  limit 
point  of  {xfc} ,  if  there  exists  one  is  a  stationary  point  of  g. 

Proof 

The  proof  is  similar  to  that  of  Theorem  3.4  and  so  we  give  only  an  outline. 
Let  0  <  6  <  1 . 

Let  7 k  be  the  smallest  eigenvalue  of  Ak  .  By  (5.3),  there  exists  7  >  0 
such  that 

7*IW|2  <  hT Akh  <  7||h||2 


$ 

i 


.7 

'.V 

Si 


K- 


m 


for  all  h  €  Sn  .  As  in  Theorm  3.4  it  is  easy  to  show  that 

w*  <  — —  =>  g{xk  -  w*pfc)  <  g(xk), 


> 


(Afc  +  7fc)(l  ~  6) 
K 


If  x*  is  a  limit  point  of  {x*}  with  xk>  — ♦  x*  ,  one  then  shows  that 

•  l|V»(x*OI|  -  0. 


But 


by  (5.3)  so  that 


liminf  ^  liminf7*y  >  0 

j  Kj 


0  =  lim  ||Vf(**>||*||Vp(x*)||.| 


(Note  that  unlike  Theorem  3.4,  Xk  =  0  for  ail  k  is  possible  so  that  A*  — ► 
0  Vg(x*)  =  0 .  This  is  a  consequence  of  our  insistence  that  for  points 


sufficiently  close  to  a  nonsingular  point,  Ak  +  A*/  must  in  fact  be  identical 
to  Ak  ,  and  hence  A k  =  0.) 

Corresponding  to  Theorem  4.5  we  have  the  following  result. 

5.5  Theorem.  Let  F:  ?Rn  — ►  SRn  ,  G:  5Rn  — ♦  the  corresponding 

M-function.  Assume  that  z*  solves  NLCP(F)  and  that  the  conditions  of 
Corollary  2.2  hold.  Assume  further  that  VF,(z),  1  <  t  <  n  is  locally  Lips- 
chitzian  at  z*  .  Then  there  exists  a  neighborhood  N(  z*  )  of  z*  such  that  if 
x°  6  N(z*)  then  {x*}  defined  by  Algorithm  5.1  satisfies  : 

(1)  xk  €  N(z*),  xk  — ►  z*  super  linearly, 

(2)  Conditions  (5.8)  and  (5.4)  of  Theorem  5.4  hold  and 

(3)  if  K  is  the  constant  given  by  (5.4),  7*  the  smallest  eigenvalue  of 
VG(2*)tVG(?*)  and  7*  >  K/2,  then  xk  — ►  z*  quadratically. 

Proof 

By  Corollary  2.2,  VG(z*)  is  nonsingular  and  G(z*)  =0.  Using  Lemma  4.2 
we  can  find  6  >  0  such  that  for  all  z  in  5  :=  S(  z*  ,  6)  =  {z  :  ||x  —  z*  }  < 
S,  VG(z)  is  nonsingular  and  sup  j|VG(z)j|  is  finite. 

Let  xk  €  S,  k  >  0 .  Then  Ak  is  positive  definite  so  that  in  Algorithm 
5.1,  >  0  and  Afc  =  0.  Hence  0  <  w*  <  1,  A*  =  0  for  all  k.  By 

[Ortega  and  Rheinboldt,  1970,  10.2.3],  z*  is  a  point  of  attraction  for  {xfc}  . 
By  [Ortega  and  Rheinboldt,  1970,  pi 24],  xk  — ►  z*  superlinearly.  It  is  easy 
to  see  that  (5.3)  and  (5.4)  hold. 


Suppose  now  that  7*  >  K/2.  Notice  that  since  A*  =  0,  we  have 


xk+l  =  xk  -ukA^Vg{xk) 

=  xk  -  WkVG^)-’  [VG(i*)r]'1VG(rfG(ifc) 

=  xk  -ukVG{xk)~lG{xk). 

One  now  shows  exactly  as  in  the  proof  of  Theorm  4.5  that  2(A*  +  7 k)/K  — * 
27  *  IK  >  1 .  Hence  by  our  choice,  for  large  k,  uk  =  1 .  Hence  Algorithm  5.1 
is  simply  the  Newton  process  and  since  VG(z)  is  Lipschitzian  in  S  (Lemma 
4.2),  it  follws  that  xk  — >  z*  quadratically.  | 

5.6  Remarks 

1.  In  Algorithm  5.1,  one  can  also  take  A*  =  A  >  0  whenever  7*  =  0 
and  computationally  this  may  be  preferable. 

2.  The  robustness  of  the  algorithm  depends  strongly  on  7* .  If  7*  is 
very  small,  then  it  is  possible  that  uk  — *  0  and  the  algorithm  fails. 

3.  Although  the  conditions  of  Corollary  2.2  are  only  sufficient  condi¬ 
tions,  the  algorithms  developed  in  this  paper  failed  to  solve  problems  in  which 
none  of  those  conditions  were  fulfilled  indicating,  perhaps,  the  sharpness  of 
those  conditions.  Nevertheless,  many  problems  in  which  the  nondegeneracy 
condition  (Corollary  2.2,  (1))  alone  failed  were  successfully  solved. 

4.  Algorithm  5.1  was  used  successfully  to  solve  Colville  test  problems 
1  and  2  ([Colville,  1968]).  Colville  problem  2  is  a  standard  difficult  test 
problem.  The  computing  time  in  each  case  compared  favorably  with  those 
obtained  by  the  use  of  Iterative  Quadratic  Programming  (IQP)  (Garcia- 


19 

Palomares  &  Mangasarian,  [1976])  using  Lemke’s  algorithm  to  solve  the 
quadratic  subproblems  :  Problem  1  was  solved  in  2.1  seconds  (0.5  seconds 
for  1QP)  and  problem  2  in  5.1  seconds  (4.5  seconds  for  IQP). 
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